Skip to content

Add compression to SBN atmospheric product generation#4649

Open
ChristopherHill-NOAA wants to merge 7 commits intoNOAA-EMC:dev/gfs.v17from
ChristopherHill-NOAA:feature/iss4614_v17_SBN-product-vol
Open

Add compression to SBN atmospheric product generation#4649
ChristopherHill-NOAA wants to merge 7 commits intoNOAA-EMC:dev/gfs.v17from
ChristopherHill-NOAA:feature/iss4614_v17_SBN-product-vol

Conversation

@ChristopherHill-NOAA
Copy link
Contributor

Description

NCO requires that the total volume of post-processing products to be transmitted through the SBN not exceed current operational levels. The generation of these products in retrospective runs of GFSv17 have marginally exceeded operational levels, and the files must be reduced in size (in lieu of the outright removal of some).

This PR simply adds a compression/packing attribute to the generation of WMO-headed atmospheric products within the script exgfs_atmos_awips_20km_1p0deg.sh, accomplished by revising the -set_grib_type option of WGRIB2 from "same" to "complex2".

This PR is intended to resolve #4614.

Type of change

  • Bug fix (fixes something broken)
  • New feature (adds functionality)
  • Maintenance (code refactor, clean-up, new CI test, etc.)

Change characteristics

  • Is this change expected to change outputs (e.g. value changes to existing outputs, new files stored in COM, files removed from COM, filename changes, additions/subtractions to archives)? YES (If YES, please indicate to which system(s))
    • GFS
    • GEFS
    • SFS
    • GCAFS
  • Is this a breaking change (a change in existing functionality)? NO
  • Does this change require a documentation update? NO
  • Does this change require an update to any of the following submodules? NO (If YES, please add a link to any PRs that are pending.)
    • EMC verif-global
    • GDAS
    • GFS-utils
    • GSI
    • GSI-monitor
    • GSI-utils
    • UFS-utils
    • UFS-weather-model
    • wxflow

How has this been tested?

The GFS workflow was cloned and built on WCOSS. A CI test only confirmed nominal functionality with the code change, as the products resulting from the low resolution run were too small for the code change to produce the necessary effect.

Offline tests executing a segment of exgfs_atmos_awips_20km_1p0deg.sh with files from products/atmos/grib2/0p25 as input resulted in a reduction to the size of the products.

It is recommended that the changes from this PR be tested in a single cycle, high-resolution test of GFSv17, to confirm its intended effect to reduce SBN product volume to the extent required by NCO.

Checklist

  • Any dependent changes have been merged and published
  • My code follows the style guidelines of this project
  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have documented my code, including function, input, and output descriptions
  • My changes generate no new warnings
  • New and existing tests pass with my changes
  • This change is covered by an existing CI test or a new one has been added
  • Any new scripts have been added to the .github/CODEOWNERS file with owners
  • I have made corresponding changes to the system documentation if necessary

Loading
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

compliance GFS V17 This issue/PR is targeting GFS V17.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants